Comparing Analytical Decision Support Models Through Boolean Rule Extraction: A Case Study of Ovarian Tumour Malignancy

The relative performances of different classifiers applied to the same data are typically analyzed using the Receiver Operator Characteristic framework (ROC). This paper proposes a further analysis by explaining the operation of classifiers using low-order Boolean rules to fit the predicted response surfaces using the Orthogonal Search Based Rule Extraction algorithm (OSRE). Four classifiers of malignant or benign ovarian tumours are considered. The models analyzed are two Logistic Regression models and two Multi-Layer Perceptrons with Automatic Relevance Determination (MLP-ARD) each applied to a specific alternative covariate subset. While all models have comparable classification rates by Area Under ROC (AUC) the classification varies for individual cases and so do the resulting explanatory rules. Two sets of clinically plausible rules are obtained which account for over one half of the malignancy cases, with near-perfect specificity. These rules are simple, explicit and can be prospectively validated in future studies.